Making Tree Ensembles Interpretable

Tree ensembles, such as random forest and boosted trees, are renowned for their high prediction performance, whereas their interpretability is critically limited. In this paper, we propose a post processing method that improves the model interpretability of tree ensembles. After learning a complex tree ensembles in a standard way, we approximate it by a simpler model that is interpretable for human. To obtain the simpler model, we derive the EM algorithm minimizing the KL divergence from the complex ensemble. A synthetic experiment showed that a complicated tree ensemble was approximated reasonably as interpretable.Comment: presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, N

The prolate dark matter halo of the Andromeda galaxy

We present new limits on the global shape of the dark matter halo in the Andromeda galaxy using and generalizing non-spherical mass models developed by Hayashi & Chiba and compare our results with theoretical predictions of cold dark matter (CDM) models. This is motivated by the fact that CDM models predict non-spherical virialized dark halos, which reflect the process of mass assembly in the galactic scale. Applying our models to the latest kinematic data of globular clusters and dwarf spheroidal galaxies in the Andromeda halo, we find that the most plausible cases for Andromeda yield a prolate shape for its dark halo, irrespective of assumed density profiles. We also find that this prolate dark halo in Andromeda is consistent with theoretical predictions in which the satellites are distributed anisotropically and preferentially located along major axes of their host halos. It is a reflection of the intimate connection between galactic dark matter halos and the cosmic web. Therefore, our result is profound in understanding internal dynamics of halo tracers in Andromeda, such as orbital evolutions of tidal stellar streams, which play important roles in extracting the abundance of CDM subhalos through their dynamical effects on stream structures.Comment: Typos corrected. Replaced to match published version. 11 pages, 7 figure

Doubly Decomposing Nonparametric Tensor Regression

Nonparametric extension of tensor regression is proposed. Nonlinearity in a high-dimensional tensor space is broken into simple local functions by incorporating low-rank tensor decomposition. Compared to naive nonparametric approaches, our formulation considerably improves the convergence rate of estimation while maintaining consistency with the same function class under specific conditions. To estimate local functions, we develop a Bayesian estimator with the Gaussian process prior. Experimental results show its theoretical properties and high performance in terms of predicting a summary statistic of a real complex network.Comment: 21 page

Non-sphericity of ultralight axion dark matter haloes in the Galactic dwarf spheroidal galaxies

Ultralight-axion (ULA) dark matter is one of the possible solutions to resolve small-scale problems, especially the core-cusp problem. This is because ULA dark matter can create a central soliton core in all dark matter haloes stemmed from the quantum pressure against gravity below the de Broglie wavelength, which becomes manifest on astrophysical scales with axion mass range $\sim10^{-22}$ eV. In this work, we apply our non-spherical dynamical models to the kinematic data of eight classical dwarf spheroidals (dSphs) to obtain more reliable and realistic limits on ULA particle mass. This is motivated by the reasons that the light distributions of the dSphs are not spherical, nor are the shapes of dark matter haloes predicted by ULA dark matter simulations. Compared with the previous studies on ULA dark matter assuming spherical mass models, our result is less stringent than those constraints due to the uncertainties on non-sphericity. On the other hand, remarkably, we find that the dSphs would prefer to have a flattened dark matter halo rather than a spherical one, especially Draco favours a strongly elongated dark matter halo caused naively by the assumption of a soliton-core profile. Moreover, our consequent non-spherical core profiles are much more flattened than numerical predictions based on ULA dark matter, even though there are still uncertainties on the estimation of dark matter halo structure. To alleviate this discrepancy, further understanding of baryonic and/or ULA dark matter physics on small mass scales might be needed.Comment: 20 pages, 12 figures, accepted for publication in MNRA

Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach

Tree ensembles, such as random forests and boosted trees, are renowned for their high prediction performance. However, their interpretability is critically limited due to the enormous complexity. In this study, we present a method to make a complex tree ensemble interpretable by simplifying the model. Specifically, we formalize the simplification of tree ensembles as a model selection problem. Given a complex tree ensemble, we aim at obtaining the simplest representation that is essentially equivalent to the original one. To this end, we derive a Bayesian model selection algorithm that optimizes the simplified model while maintaining the prediction performance. Our numerical experiments on several datasets showed that complicated tree ensembles were reasonably approximated as interpretable.Comment: 21 page

Structural properties of non-spherical dark halos in Milky Way and Andromeda dwarf spheroidal galaxies

We investigate the non-spherical density structure of dark halos of the dwarf spheroidal (dSph) galaxies in the Milky Way and Andromeda galaxies based on revised axisymmetric mass models from our previous work. The models we adopt here fully take into account velocity anisotropy of tracer stars confined within a flattened dark halo. Applying our models to the available kinematic data of the 12 bright dSphs, we find that these galaxies associate with, in general, elongated dark halos, even considering the effect of this velocity anisotropy of stars. We also find that the best-fit parameters, especially for the shapes of dark halos and velocity anisotropy, are susceptible to both the availability of velocity data in the outer regions and the effect of the lack of sample stars in each spatial bin. Thus, to obtain more realistic limits on dark halo structures, we require photometric and kinematic data over much larger areas in the dSphs than previously explored. The results obtained from the currently available data suggest that the shapes of dark halos in the dSphs are more elongated than those of $\Lambda$CDM subhalos. This mismatch needs to be solved by theory including baryon components and the associated feedback to dark halos as well as by further observational limits in larger areas of dSphs. It is also found that more diffuse dark halos may have undergone consecutive star formation history, thereby implying that dark-halo structure plays an important role in star formation activity.Comment: Typos corrected. 16 pages, 12 figures. Accepted for publication by Ap

A common surface-density scale for the Milky Way and Andromeda dwarf satellites as a constraint on dark matter models

In an attempt to place an explicit constraint on dark matter models, we define and estimate a mean surface density of a dark halo within a radius of maximum circular velocity, which is derivable for various galaxies with any dark-matter density profiles. We find that this surface density is generally constant across a wide range of maximum circular velocities of $\sim$ 10 to 400 km s$^{-1}$, irrespective of different density distribution in each of the galaxies. This common surface density at high halo-mass scales is found to be naturally reproduced by both cold and warm dark matter (CDM and WDM) models, even without employing any fitting procedures. However, the common surface density at dwarf-galaxy scales, for which we have derived from the Milky Way and Andromeda dwarf satellites, is reproduced only in a massive range of WDM particle masses, whereas CDM provides a reasonable agreement with the observed constancy. This is due to the striking difference between mass-concentration relations for CDM and WDM halos at low halo-mass scales. In order to explain the universal surface density of dwarf-galaxy scales in WDM models, we suggest that WDM particles need to be heavier than 3 keV.Comment: Typos corrected. 5 pages, 3 figures. Accepted for publication by ApJ Letter

A Tractable Fully Bayesian Method for the Stochastic Block Model

The stochastic block model (SBM) is a generative model revealing macroscopic structures in graphs. Bayesian methods are used for (i) cluster assignment inference and (ii) model selection for the number of clusters. In this paper, we study the behavior of Bayesian inference in the SBM in the large sample limit. Combining variational approximation and Laplace's method, a consistent criterion of the fully marginalized log-likelihood is established. Based on that, we derive a tractable algorithm that solves tasks (i) and (ii) concurrently, obviating the need for an outer loop to check all model candidates. Our empirical and theoretical results demonstrate that our method is scalable in computation, accurate in approximation, and concise in model selection

Estimation of low-rank tensors via convex optimization

In this paper, we propose three approaches for the estimation of the Tucker decomposition of multi-way arrays (tensors) from partial observations. All approaches are formulated as convex minimization problems. Therefore, the minimum is guaranteed to be unique. The proposed approaches can automatically estimate the number of factors (rank) through the optimization. Thus, there is no need to specify the rank beforehand. The key technique we employ is the trace norm regularization, which is a popular approach for the estimation of low-rank matrices. In addition, we propose a simple heuristic to improve the interpretability of the obtained factorization. The advantages and disadvantages of three proposed approaches are demonstrated through numerical experiments on both synthetic and real world datasets. We show that the proposed convex optimization based approaches are more accurate in predictive performance, faster, and more reliable in recovering a known multilinear structure than conventional approaches.Comment: 19 pages, 7 figure

Formation of massive globular clusters with dark matter and its implication on dark matter annihilation

Recent observational studies of $\gamma$-ray emission from massive globular clusters (GCs) have revealed possible evidence of dark matter (DM) annihilation within GCs. It is, however, still controversial whether the emission comes from DM or from milli-second pulsars. We here present the new results of numerical simulations, which demonstrate that GCs with DM can originate from nucleated dwarfs orbiting the ancient MW. The simulated stripped nuclei (i.e., GCs) have the central DM densities ranging from 0.1 to several ${\rm M_\odot pc^{-3}}$, depending on the orbits and the masses of the host dwarf galaxies. However, GCs born outside the central regions of their hosts can have no/little DM after their hosts are destroyed and the GCs become the Galactic halo GCs. These results suggest that only GCs originating from stellar nuclei of dwarfs can possibly have DM. We further calculate the expected $\gamma$-ray emission from these simulated GCs and compare them to observations of $\omega$ Cen. Given the large range of DM densities in the simulated GCs, we suggest that the recent possible detection of DM annihilation from GCs should be more carefully interpreted.Comment: 5 pages, 4 figures, to be published in MNRA